Data Understanding

CLOSING PRICE cenderung naik trend nya

Types of Volatility and How They are Measured

(Source: https://www.investopedia.com/terms/v/vix.asp)

Volatility does not measure the direction of price changes of a financial instrument, merely its dispersions over a certain period of time. The more dramatic the price swings are in that instrument, the higher the level of volatility, and vice versa. High volatility is associated with higher risk, and low volatility low risk.

Volatility can either be historical or implied; both are usually expressed in percentage terms.

Historical Volatility (HV) or Realized Volatility is the actual volatility demonstrated by the underlying over a period of time. Realized Volatility is commonly calculated as the standard deviation of price returns, which is the dollar change in price as a percentage of previous day's price. Implied volatility (IV), on the other hand, is the level of volatility of the underlying that is implied by the current option price. (The main focus of this project is NOT implied volatility, which traditionally can be derived from option pricing models such as the Black Scholes Model).

Further research indicates that for practicality purposes it's preferable to use the log returns especially in mathematic modeling, because it helps eliminate non-stationary properties of time series data, and makes it more stable:

LOG RETURNS FORMULA:

There's another advantage to log returns, which is that they're additive across time:

The goal here is to forecast the average realized volatility of the next n_future 7 days using a number of immediate past/historical datapoints (n_past).

The reason I selected 30 days is because 7 days seems too noisy to observe meaningful patterns, while longer intervals seem to smooth the volatility down significantly and tend to mean-revert.

Using interval window of 30 days would also help avoid wasting too many datapoints at the beginning of the dataset.

Since I am currently using aN INTERVAL_WINDOW of 30 and a horizon n_future of 7, the volatilty of first 30 values as well as the last 7 values of the dataframe would be NaN, and therefore need to be dropped from the dataset.

Daily Volatility Grouped by Month

SENTIMEN vaksin Covid-19 yang sedang diuji klinis oleh Biofarma bekerjasama dengan perusahaan Tiongkok Sinovax, ikut mendorong kenaikan saham PT Darya-Varia Laboratoria Tbk. (DVLA)

OVER REAKSI: Ia menegaskan, pasar selalu over reaksinya sehingga dalam bursa selalu ada circuit breaker. Tetapi pelan-pelan pasar akan melakukan koreksi jika secara teknikal harga saham sudah dianggap terlalu tinggi.

GORENGAN: Ia meriset, saham-saham yang terbang dalam pekan ini sebagian besar adalah portfolio Asabri dan Jiwasraya. Jadi cara merestorasi kerugian mereka, harga sahamnya dinaikkin. "Kebetulan ada momentum vaksin, sehingga bisa dikatakan tidak menggoreng tetapi karena sentimen," jelasnya.

It can be observed that:

1)volatility has consistently ranged around number 0.05 but somehow reached some of its higher points in the months of October/November historically

Daily Volatility Grouped by Year

2018 kepemilikan saham dvla di etana biotechnology terdilusi

Daily Volatility Distribution

The distribution of daily realized volatility is lightly right skewed, with a small number of larger values spreaded thinly on the right.

A skewed right distribution would have smaller median compared to mean, and mode smaller than median (mode < median < mean).

Returns/Log Returns Stationarity Checking

Using a significant level alpha of 0.05, p-value for both Returns and Log Returns are significantly smaller than alpha, which means there's enough evidence to reject the Null Hypothesis.

--> Returns and Log returns are both not dependent on time/trend

Train-Validation-Test Splits

I would split the dataset into 3 parts as follows:

the most recent 30 usable datapoints would be used for final testing - approx. 1.2% 1 full year (365 days) for validation and model tuning during training - approx. 14.7% and the remaining for training - approx. 84.1%

The target here would be vol_future which represents the daily realized volatility of the next n_future days from today (average daily volatility from t + n_future - INTERVAL_WINDOW + 1 to time step t + n_future).

For example, using an n_future value of 7 and an INTERVAL_WINDOW of 30, the value that I want to predict at time step t would be the average daily realized volatility from time step t-22 to time step t+7

Feature Normalization

As I am going to build various types of models based on different algorithms with slightly different assumptions using different types of parameters, it's probably better to normalize the volatilities using MinMaxScaler to standardize performance comparison among different models.

Train-Validation-Test Visualization

Modeling

Performance Metrics

Usually with financial time series, if we just shift through the historic data trying different methods, parameters and timescales, it's almost certain to find to some strategy with in-sample profitability at some point. However the whole purpose of "forecasting" is to predict the future based on currently available information, and a model that performs best on training data might not be the best when it comes to out-of-sample generalization (or overfitting). Avoiding/Minimizing overfitting is even more important in the constantly evolving financial markets where the stake is high.

The 2 main metrics I'd be using are RMSPE (Root Mean Squared Percentage Error) and RMSE (Root Mean Square Errors) with RMSPE prioritized. Timescaling is very important in the calculation of volatility due to the level of freedom in frequency/interval window selection. Therefore I think RMSPE would help capture degree of errors compared to desired target values better than other metrics. Also RMSPE would punish large errors more than regular MAPE (Mean Absolute Percentage Error), which is what I want to do here.

RMSE and RMSPE would be tracked across different models' performance on validation set forecasting to indicate their abilities to generalize on out-of-sample data.

Model #1 - Baseline Model

One of the essential characteristics of Volatility is it's mean-revert over the long term. Therfore my first baseline model his would be a very simple one that only outputs the average current realized volatility of the whole training set as predictions everything.

Model #2 - Random Walk Naive Forecasting

A commonly known fact about volatility is that it tends to be autocorrelated, and clusters in the short-term. This property can be used to implement a naive model that just "predicts" future volatility by using whatever the daily volatility was at the immediate previous time step.

In this case, I'll use the average daily volatility of the most recent INTERVAL_WINDOW as predictions for the next 7 days, which is essentially using vol_current at time step t and prediction for vol_future at time step t.

Model #3 - GARCH

(Reference: http://users.metu.edu.tr/ozancan/ARCHGARCHTutorial.html)

GARCH stands for Generalized Autoregressive Conditional Heteroskedasticity, which is an extension of the ARCH model (Autoregressive Conditional Heteroskedasticity).

GARCH includes lag variance terms with lag residual errors from a mean process, and is the traditional econometric approach to volatility prediction of financial time series.

Mathematically, GARCH can be represented as follows:

in which is variance at time step t and is the model residuals at time step t-1

GARCH(1,1) only contains first-order lagged terms and the mathematic equation for it is:

where , and sum up to 1, and is the long term variance.

(Reference: Sinclair)

GARCH is generally regarded as an insightful improvement on naively assuming future volatility will be like the past, but also considered widely overrated as predictor by some experts in the field of volatility. GARCH models capture the essential characteristics of volatility: volatility tomorrow will be close to what it is today (clustering), and volatility in the long term will probably mean revert (meaning it'd be close to whatever the historical long-term average has been).

Basic GARCH

Analytical-based Forecasting

(Source: https://arch.readthedocs.io/en/latest/univariate/univariate_volatility_forecasting.html)

I am implementing rolling one-step forecasting here by refitting the model to ALL the datapoints available up to a certain time step before generating prediction for the next n_future days. For example, to make prediction on vol_future at time step t, I would fit the model to all the returns available up to time step t and then obtain the average forecasted volatility for a horizon of n_future = 7.

The volatility forecasts from the GARCH models using percent returns as input are on a totally different scale compared to the volatilities calculated from log returns. Therefore I am going to normalize the forecasted volatility based on the model's conditional_volatility output from the training data, and only comparing the scaled versions of volatilities on the Validation set.

DI RMSE bahkan garch lebih gede drpd mean baseline. auto drop

Model #4 - Neural Networks

While GARCH remains the gold standard for volatility prediction within traditional financial institutions, there has been an increasing numbers of professionals and researchers turning to Machine Learning, especially Neural Networks, to gain insights into the financial markets in recent years.

(Reference: Sinclair (2020))

Traders' theory of the market being inherently efficient (Efficient Market Hypothesis or EHM) states that share prices reflects all information and consistently outperforming the overall market is impossible. The more efficient a market is, the more random and unpredictable the returns will be, and thus a perfectly efficient market will be completely unpredictable.

There are other arguments against EMH, and ones of the most prominent one is based on Behavioral Finance: compared to the human history of 200,000 years, the market has not been around for that long. For example, equity options have only been traded in liquid, transparent market since the CBOE opened in 1973; and the average lifetime of an S&P500 company is approx. 20 years. It means that some psychological tendencies of human beings have 200,000 years of evidence behind them, and that a lot of the movements of the markets that were driven by participants' behaviors will likely repeat itself at a later point. Therefore the market system cannot be totally random, it must have some patterns. Those patterns are extremely difficult to exploit due to the multitude of factors that interact and drive the market.

And what could be better than Neural Networks for that?

BTS SINI

BEST RESULT SO FAR

multivariate Long Short-Term Memory (LSTM)

(Reference: Geron(2019))

There's a class of Neural Networks called Recurrent Neural Networks (RNN) that can "predict the future". RNN works well with time series data, such as stock prices, and can even process sentences, documents, audio samples as inputs. Another application of RNN is in autonomic driving systems, where they're used to anticipate vehicle trajectories and help avoid accidents.

COBA MULTI

Feature Scaling

FINAL

For this final model, I'll re-create the same architecture as model #5 (Multivariate 2 Bidirect LSTM layers (20/10 units) | n_past=30 | batch=64) and a lookback window n_past = 14 and batch_size = 64. However, I'll extend the training to include the validation set as well. This is important in time series data since the more recent time steps would most likely carry more relevant information for successful forecasting than data from a year ago.